In order to perform bidirectional video communication such as videotelephony, delay should be low. This “delay” indicates a time measured from when a video sending side of a system acquires a picture using a camera or the like, to when a video decoding side of the system displays this acquired picture.
In addition, a “picture” denotes a frame or a field in input video.
To such application, a video coding technique is applied so as to reduce the bandwidth frequency required for video transmission. Among the total delay time, video coding technique mainly affects (i) time required for encoding executed on the video sending side, (ii) time required for buffering of encoded data on the video receiving side, and (iii) time required for decoding of the encoded data on the video receiving side. In order to reduce the delay time for the encoding/decoding processes, speed for these processes should be increased. Additionally, in order to reduce the encoded data buffering time on the video receiving side, employment of intra slices (explained later) and a quantization control technique suitable therefor is effective.
In general video coding, an intra-type picture is inserted regularly for a purpose of refreshing. However, in comparison with an inter-type picture, the intra-type picture has a very large amount of code, which requires a long buffering time for the encoded data on the video receiving side, thereby the delay time is increased.
Instead of using the intra picture, the video encoding that employs intra slices realizes refreshing by moving, in each picture, a band in a longitudinal or lateral direction, instead of using intra pictures, where the band forcibly subjects all macroblocks in the band to intra encoding. In this process, inter encoding may be performed in each region (in the picture) other than the intra slices, where such a region will be called a “non intra slice region”.
According to such a method, the amount of code generated for each picture can almost be constant, thereby the buffering time can be reduced.
When arranging the above-described band in a longitudinal direction, it may not be called the “intra slice” (and may be called an “intra column”). However, for convenience of explanation, the present specification uses “intra slice” for either case of moving the band in a longitudinal or lateral direction.
FIG. 9 is a diagram showing a concept of a case that provides an intra slice arranged in a longitudinal direction. In the shown example, the intra slice moves from the left end to the right end in the relevant picture, where time for the number of pictures among which the position of the intra slice moves in one period (i.e., the number of pictures for which the intra slice moves from the left end to the right end for the pictures) is called an “intra slice period”.
when the video receiving side cannot accurately generate a decoded picture due to packet loss or the like, if the intra slice moves from the left side to the right side of the picture after such inaccurate decoding, then accurate decoded images are obtained after that (i.e., refreshing has been performed). That is, after the packet loss or the like occurs, refreshing has been completed within twice the intra slice period.
When using the intra slice, the amount of generated code considerably differ between the intra slice region and the non intra slice region. Therefore, a quantization control method that can allocate an appropriate amount code to each region is required.
Patent-Document 1 discloses a technique that implements such a technique.
In the technique disclosed in Patent-Document 1, (i) an amount of code generated for an intra slice region in a picture immediately before an encoding target picture and (ii) an average quantization step (these items (i) and (ii) will be collectively called “encoding information”) are used to compute complexity index Xi for the relevant region.
Similarly, encoding information for a non intra slice region in the picture immediately before the encoding target picture is used to compute complexity index Xp for the relevant region.
Based on Xi and Xp, a target amount T of code for the encoding target picture is divided into a target amount Ti of code for the intra slice region and a target amount Tp of code for the non intra slice region. Then, based on Ti and Tp, the quantization step for each macroblock in the picture is determined.
The above method causes the following problems.
Generally, the shorter the intra slice period (i.e., the smaller the size of the intra slice region in each picture), the lower the image quality. Therefore, in most cases, the intra slice period is set to 0.5 sec or longer. When the intra slice period is short, the area of the intra slice becomes considerably small. For example, if the intra slice period is 0.5 sec and 30 pictures are present in each second, then the area of the intra slice region is one-fourteenth of the area of the non intra slice region.
Therefore, the property of video may considerably differ between the intra slice in an encoding target picture and the intra slice in a picture immediately before the encoding target picture. In such a case, the complexity index Xi is not accurate, which degrades the image quality.
In such circumstances, it can be easily anticipated to use encoding information for the intra slice in an earlier picture than the encoding target picture by one intra slice period (i.e., a picture that has the same intra slice position (on the picture) as that of the encoding target picture). However, since such an earlier picture is temporally far away from the encoding target picture, if an object has moved, the property of the intra slice has considerably changed, which also causes degradation in the image quality.
In order to solve the above problem, a technique disclosed in Patent-Document 2 may be combined with the technique of Patent-Document 1.
Although the technique of Patent-Document 2 is not applied to a case using the intra slice, it corrects a complexity index of the encoding target picture when computing the complexity index by using encoding information of a picture immediately before the encoding target picture, where the correction is performed based on activity measures of the encoding target picture and the picture immediately before. The activity measure represents a characteristic feature of each image in a manner such that the finer (i.e., including edge portions) the texture of each target block, the larger the activity measure. The relevant technique is effective when the property slightly differs between the encoding target picture and the picture immediately before, for example, during a fade-in or fade-out period.
If the technique of Patent-Document 2 is applied to the complexity index computation for the intra slice in the technique of Patent-Document 1, image quality can be improved when the property of the intra slice differs between the encoding target picture and an earlier picture than the encoding target picture by one intra slice period.